1. Technical Field
Example embodiments of the present invention relate in general to a stereo vision system and more specifically to a system and method for performing distortion correction and calibration using a pattern projected by a projector.
2. Related Art
In recent years, in order to reproduce a natural three-dimensional (3D) image, an image synthesis method using a depth image has been widely used. The depth image is invisible to a viewer's eyes but is important information for determining the image quality of a synthesized image, and therefore it is important to acquire an accurate depth image.
The depth image may be acquired by a method of using an active depth sensor and a method of using a passive depth sensor. The method using the active depth sensor may directly acquire depth information using a physical sensor device (infrared sensor, or the like), whereas the method of using the passive depth sensor may calculate depth information from images obtained through at least two cameras.
In particular, stereo matching may acquire the depth information by finding pixels coinciding with pixels of one image from two images of the same scene obtained from mutually different viewpoints, from the other images. However, stereo vision has a large amount of computation due to complexity of an algorithm thereof, and there are some constraints in order to reduce the large amount of computation. Among the constraints, there is an epipolar line constraint in which matching points in the stereo vision are present on the same epipolar line in each image. Thus, most systems for acquiring the depth image have a hardware device or a software method which can adjust distortion correction and calibration with respect to cameras in order to satisfy the epipolar line constraint.
In a technology proposed by Caltech among the technologies related to the above description, a checkerboard with a predetermined pattern may be photographed in various angles and circumstances, and required parameters may be obtained through images about the photographed checkerboard.
FIG. 1 is a pre-treatment method proposed by Caltech, and in FIG. 1, an example in which rectangular vertices, shapes of the rectangular vertices, and an equation of a plane of a checkerboard constituted of the vertices and the shapes are extracted from images obtained by photographing a rectangular plane (checkerboard) having a chessboard-shaped pattern using a stereo camera to thereby match a calibration state of a left image and a right image is shown.
However, in this method, accuracy may be increased by an increase in the number of images obtained by photographing the checkerboard, and in particular, when it fails to photograph images about various angles and regions, an error may occur with respect to a region in which the checkerboard is not photographed.
FIG. 2 is an exemplary view illustrating an image of a case in which the technology proposed by Caltech is applied. Referring to FIG. 2, when applying parameters according to an image with respect to a checkerboard concentrated in the center of a photographed image, there is no problem in calibration and distortion correction of the center portion of the image, but it can be seen that severe distortion occurs at the edges of the image.
In addition, there is no method for completely fixing the cameras, and therefore, when the calibration state of the cameras is consequently shifted again due to slight distortion or the like in the use of the cameras, a process in which the images of the checkerboard are acquired again for the purpose of correction and parameters are newly extracted from the acquired images should be performed.
That is, in the method using an existing checkerboard, the accuracy may be increased by obtaining a large number of images in various locations of the entire image region. In addition, rectangular vertices that cannot be automatically found should be input directly by a user, and when the calibration is shifted, there is a problem that calibration should be performed again, or the like.